COMMENT ⊗   VALID 00016 PAGES
C REC  PAGE   DESCRIPTION
C00001 00001
C00003 00002	\init{
C00004 00003	\chapbegin{8}		% Here beginneth Chapter 8
C00006 00004	\sectionbegin[2]{Review of major issues}\par
C00012 00005	\subsectionbegin[2.3]{Knowledge Acquisition}
C00027 00006	\subsectionbegin[2.4]{Strategies}
C00034 00007	\sectionbegin[3]{Global limitations}
C00044 00008	\sectionbegin[4]{The other themes some speculations}
C00046 00009	\subsectionbegin[4.1]{Why Write a Program:  Two Views}
C00062 00010	\dimsectionbegin{Theme 2:  Knowledge in programs should be explicit and accessible.}
C00070 00011	\dimsectionbegin{Theme 3:  Programs can be given access to and an
C00080 00012	\dimsectionbegin{Theme 4:  Programs can be self-understanding.}
C00087 00013	\dimsectionbegin{Theme 5:  A representation can usefully be more than a
C00093 00014	\dimsectionbegin{Theme 6:  A program can have some grasp on its own complexity.}
C00098 00015	\dimsectionbegin{Theme 7: Programs can be self-adjusting.}
C00113 00016	\sectionbegin[5]{Projections}
C00118 ENDMK
C⊗;
\init{
\input cmpdes
\input temacs
\def\draft{T}
\titlepage
\newpage
\setcount0 233
\parttitle{TEIRESIAS: META-LEVEL KNOWLEDGE IN KNOWLEDGE-BASED SYSTEMS}
}
\baselineskip 12pt
\trace'344
\chapbegin{8}		% Here beginneth Chapter 8
\chaptertitle{CONCLUSIONS}
\rjustline{\:B Conclusions}
\tenpoint
\vskip 13.7pc

\epigraph{Alas, how terrible is wisdom when it brings no profit to the man
that's wise!
This I knew well, but had forgotten it; else I would not have come here.}
\author{Sophocles}
\source{Oedipus the King, lines 312--313}
\epigraphend


\sectionbegin[1]{Introduction}
The last five chapters have explored a number of different problems
encountered in building {\TEIRESIAS}, a program intended to establish a link between a
human expert and a computer consultant.  Those problems and solutions are reviewed
briefly here and  considered in the light of meta-level knowledge
as a tool for knowledge base construction, maintenance, and use.  This is
followed by a discussion of some of the global limitations and shortcomings that
arise from the attempt to link the expert and program.  Finally, we return to
the alternative set of themes listed in chapter 1, reconsidering some of this
work in those terms and using them as a basis for speculation about future
directions.

\newpage
\sectionbegin[2]{Review of major issues}\par
\subsectionbegin[2.1]{Forms of Meta-Level Knowledge}
\index{meta-level knowledge}Figure 8--1 reviews the three major forms of meta-level
knowledge developed in previous chapters.  The rule models describe the
{\sl content} of inference rules in the knowledge base, making clear the global
trends in those rules and providing useful assistance in acquiring new
rules.  The data structure schemata describe the {\sl structure} of the
conceptual primitives used in expressing rules and offer a basis for the
acquisition of new primitives.  Finally, the meta-rules describe how to {\sl use}
object-level rules and function as strategies to guide invocation of those
rules.

	One additional, less developed source of meta-level knowledge is
the function templates (chapter 2), which describe part of the structure of
inference rules and, like the rule models, are used in acquiring new rules.

\null
{\tt\nofill
   FORM              DESCRIBES               USE


Rule models         CONTENT of           Acquisition of
                    inference rules      new inference rules

Data structure      STRUCTURE of         Acquisition of 
schemata            conceptual           new conceptual
                    primitives           primitives

Meta-rules          USE of object-       Guide use of 
                    level rules          object-level knowledge
\endnofill}

\figtitle 8-1{The three forms of meta-level knowledge.}


\subsectionbegin[2.2]{Explanation}
Chapter 3 discussed efforts to enable {\TEIRESIAS} to explain the reasoning
of a performance program to audiences that range from an expert acquainted
with the program to a student with minimal experience in the field.  The
basic steps in building the explanation facility involved
(a)  determining the
program operation that was to be considered primitive, (b)  augmenting the
performance program to leave behind a trace of behavior in terms of this
operation, (c)  finding a framework in which that trace could be understood,
and (d)  designing a program that would allow the user to examine the
behavior trace in terms of the framework.  This supplied a mechanism for
exploring past actions; future actions of the system were explored by means
of code in the explanation system which simulated the control structure of
the performance program.

	This approach to explanation relies on the assumption that a recap
of program operations can be a reasonable basis for generating
explanations.   This was seen to suggest the areas of greatest applicability
as well as imply a number of limitations.  In particular, it seems best
suited to programs whose primary mode of computation is symbolic reasoning
rather than numeric operations.

	The primary shortcomings in the current implementation were a
failure to factor in the state of the viewer's knowledge, a lack of a
general notion of what an explanation is, and a lack of ability to
represent control structures.  The latter becomes particularly evident when
changes are made to the control structure of the performance program, since
this often requires significant changes to the explanation program.  We
speculated about the possibility of a representation of control structures
that emphasized intentional information, which, combined with a
formalization of the concept of explanation, might make the system far more
flexible and general.  It might, ideally, be possible for the system to
examine its own code, generating explanations on the basis of what it found
there.

\subsectionbegin[2.3]{Knowledge Acquisition}
\index{interactive transfer of expertise}Knowledge acquisition
was described as a process of interactive
transfer of expertise from an expert to a performance program, in which {\TEIRESIAS}'s
task was to ``listen'' as attentively and intelligently as possible.  The
process was set in the context of a shortcoming in the knowledge base, as an
aid to both the expert and the system.  This context provides the expert
with a useful organization and focus.  He is not simply asked to describe
all he knows about a domain.  He is instead faced with a specific
consultation whose results he finds incorrect and has available to him a set of
tools that will allow him to uncover the extent of the system's knowledge
and the rationale behind its performance.  His task is then to specify the
particular difference between the system's knowledge and his own that
accounts for the discrepancy in results.  The system relies on the context
of the error to form a set of expectations about the character of the
information that will be forthcoming.  This leads to better comprehension
of the expert's statement of that information and provides a number of
checks on its content that insure it will in fact repair the problem
discovered.  In a single phrase, {\sl interactive transfer of expertise in the
context of a shortcoming in the knowledge base} characterizes our approach
to the problem and suggests the source of many of the system's abilities.

\subsectionbegin[2.3.1]{Acquiring New Rules}
\index{model-based understanding}\index{recognition}New rule acquisition
was seen in terms of model-directed
understanding and a recognition-oriented approach to comprehension.  This
means that the system has some model of the content of the signal it is
trying to interpret and uses this to help constrain the set of
interpretations it will consider.  In our case, the model took the form of
rule models, constructs which offer a picture of a ``typical'' rule of a
given type.  These models are assembled by the system itself, using a primitive,
statistically oriented form of concept formation to produce abstract
descriptions of syntactic regularities in subsets of rules.

	As noted, the context provided by the process of tracking down the\index{expectations}
error in the knowledge base makes it possible for {\TEIRESIAS} to form expectations
about the character of the new rule.  These expectations are expressed by
selecting a specific rule model.  The text of the new rule is then allowed
to suggest interpretations (a bottom-up, data-directed process),
but these are constrained and evaluated for likely validity by
reference to the rule model (a top-down, hypothesis-driven process).  It is
the intersection of these two information sources, approaching the task
from different directions, that is responsible for much of the system's
performance.

	Further application of the model-directed formalism is seen in {\TEIRESIAS}'s
ability to ``second guess'' the expert.  Since it has a model of the
knowledge base---the rule models---it can tell when something ``fits'' in the
knowledge base.  It is the occurrence of a partial match between the new
rule and the selected rule model that prompts {\TEIRESIAS} to make suggestions to the
expert.  This idea of an unmet expectation is not specific to
the current organization or structure of the rule models and can be
generalized to cover any aspects of a representation about which
expectations can be formed.

	Several implications were seen to follow from the fact that {\TEIRESIAS}
constructs the rule models from current contents of the knowledge base.
Since the process is automated, the expert never has to enter models by
hand; he may even be unaware of their existence.  Moreover, unlike most
other model-based systems, new models are constructed on the basis of past
experience, since rules learned previously are used in forming new models.
Since the models are updated as each new rule enters the knowledge base,
the model set is kept current, evolving with the growing knowledge base
and reflecting the shifting patterns it contains.

	Other implications follow from the fact that these models give the
system an abstract picture of its own knowledge base.  It means that, in a
rudimentary way, the system ``knows what it knows, and knows where it is
ignorant.''  It can answer questions about the content of its knowledge base
by ``reading'' a rule model, giving a picture of global structure of its
knowledge about a topic.  Since the models are ordered on the basis of an
empirically defined ``strength,'' the system can also give some indications
about possible gaps in its knowledge.

	Finally, the coupling of model formation with the model-directed
understanding process offers a novel form of closed-loop behavior. Existing
rule models are used to guide the acquisition process, the new rule is
added to the knowledge base, and the relevant rule models are recomputed.
Performance of the acquisition routines may thus conceivably be improved on
the very next rule.

	In summary, {\TEIRESIAS} constructs models of the knowledge base;
it updates those models in response to changes, keeping them
an accurate reflection of the current knowledge base; and it then uses
them to aid in the acquisition of new knowledge.


\subsectionbegin[2.3.2]{Acquiring New Conceptual Primitives}
\index{knowledge about representations}The primary issue
here is the representation and use of knowledge
about representations.  The schemata and associated structures offer a
language for the expression of the knowledge and a framework for its
organization.  There are three levels to that organization: ({\sl i}) the
individual schema is the fundamental unit of organization and is a
record-like structure that provides the basis for assembling a variety of
information about a particular representation; ({\sl ii}) the schema network
is a generalization hierarchy that indicates the existing categories of
data structures and relationships between them; and ({\sl iii}) the slotnames
and slotexperts that make up a schema deal with specific representation
conventions at the programming language level.

	Unlike standard records, however, the schemata and all associated
structures are a part of the system itself and are available for examination and
reference.  They also have the ability to describe a certain amount of
variability in structure description.

	The process of acquiring a new conceptual primitive strongly
resembles the creation of a new instance of a record, but has been extended
in several ways.  It has been made interactive, to allow the expert to
supply information about the domain; the dialog is couched in high-level
terms, to make it comprehensible to a nonprogrammer; and the whole process
has been made as easy and ``intelligent'' as possible, to ease the task of
assembling large amounts of knowledge.

	This approach involves viewing representational primitives as
extended data types and constructing the appropriate schema for each of
them.  That is, the language for describing representations was used to
formalize a range of information about the performance program's
representations.

	The generality of this approach results from a stratification and
isolation of different varieties of knowledge at different levels: 
Instances of individual schemata form the collection of domain-specific
knowledge; the schemata themselves define a base of representation-specific
information; while the schema-schema supplies a small foundation of
representation-independent knowledge.  This stratification makes it
possible for the system to acquire both new instances of existing
representations (as in learning about a new organism) and new types of
representation (as in the acquisition of a new schema), using a single
formalism and a single body of code.

	Finally, it was noted that the same motivation was responsible for
both the schemata and the recursive application of the idea to produce the
schema-schema.  The schemata were designed to automate the handling of the
large number of details involved in the creation and management of data
structures.  But they themselves were sufficiently complex,  detailed
data structures that it was useful to have a similar device for their
construction and management.  This resulted in the creation of the
schema-schema, and it, along with a small body of associated structures,
forms a body of representation-independent knowledge from which a knowledge
base can be constructed.

\subsectionbegin[2.4]{Strategies}
The final form of meta-level knowledge explored was the concept of
a strategy, defined as knowledge about the use of knowledge.  This
definition was extended to include the possibility of an arbitrary number
of levels of strategies, each of which can direct the use of the
information at the next lower level.  We considered the possible building
blocks for a strategy language and speculated about the source of the
conceptual primitives from which such a language might be built.  The
resulting framework was seen to offer a reasonably general view, one that
can help to organize and make explicit strategy knowledge that is
otherwise often embedded subtly in program code.

	We explored the character of problems for which this approach is
useful, noted that it is most applicable to what are called ill structured
problems and found that it offers the greatest advantage for programs with large
knowledge bases subject to frequent change.

	Meta-rules were described as one example of these ideas.  They were
seen to offer a convenient mechanism for the expression of strategies and
proved capable of guiding program performance without introducing
unreasonable overhead.  The organization of knowledge they provide appears
novel, in guiding heuristic search without being part of a search
algorithm.  The search routine itself is very simple; the
``intelligence'' is organized around and stored in the goal tree itself.
This means that at any point where the system has to choose a path through
the tree, there may possibly be information stored at that branch point
available to guide it.

	Meta-rules illustrated several interesting issues when considered
as an invocation mechanism.  First, the fact that meta-rules refer to
object-level rules by reference to their content rather than by name was\index{content-directed invocation}
termed content-directed invocation and found to be a generalization of
some of the traditional mechanisms of knowledge source invocation.  This\index{expressiveness}\index{validity}
technique offers a greater degree of expressiveness and validity than is\index{flexibility}
typically available and was seen to provide a high level of flexibility.
Second, meta-rules offer a framework in which the user can define his own\index{generalized invocation criteria}
invocation criteria, leading to the idea of generalized invocation
criteria.  This frees the programmer from the restriction of using only
those invocation criteria that are predefined and embedded in the programming
language in use.  He can instead define the set of criteria he wishes to
use and can, as well, write programs capable of choosing from among that set
at execution time.  Third, the presence of multiple levels of meta-rules
means that the invocation criteria (meta-rules at one level) are also
treated as data objects (by meta-rules at the next higher level).  This
means that the system has a primitive capability to reason about the
control structure it should use.

	The limitations of this approach and its costs were also
considered.  While it is possible at present to implement it in simple
ways, sophisticated use requires advances in the field of program
understanding.  It also imposes a computational overhead.  While some of
this can be relegated to a background pre-compilation phase, the system
cannot then create rules dynamically during execution of the performance
program.  As noted above, the approach seems to offer the most substantive
advantages when dealing with large knowledge bases with numerous
strategies, in systems subject to frequent change.  It is thus appropriate
for programs where the size of the knowledge base and the necessity of an
incremental approach to competence require the ability to assemble large
amounts of knowledge and the ability to make numerous alterations to it.

\sectionskip
\sectionbegin[3]{Global limitations}
The fundamental problem we started out to consider was the creation
of an intelligent link between a domain expert and a high-performance
program.  This provided a large collection of topics for discussion, and
previous sections have noted the shortcomings and limitations of each of
the individual solutions pro\-posed.  But what of this basic notion of
linking the expert and program?  What limitations might there be in that
whole approach, and what limitations may arise from the particular
solutions proposed and implementations described?

	First, the current version of {\TEIRESIAS} is  still quite rough.  It
has not yet been given to any users for rigorous testing.  It needs a good
deal of polish and more attention to user convenience before it becomes a
truly useful tool.  Some suggestions along these lines were noted in
earlier chapters, but there is a more basic question concerning the nature
of the tools we have assembled.  Some are conceptually neat, but are they
what an expert building a large knowledge base would really find most
useful?

\index{second guessing}For instance, is
{\TEIRESIAS}'s ability to second guess an expert's
new rule going to help, or will it only get in the way?  Can it be made
``smarter''?  Could the system learn, for example, to recognize the fact that
the expert is entering a whole new sequence of rules and that none of its
current expectations are likely to be met?\ffnote{In this sense the system is 
currently a bit like an overly eager student who is always trying to out-guess
the instructor.  There really ought to be a way to tell it to ``keep quiet
and listen.''} In more general terms, we have stressed the advantages of
debugging in context, but are there ways of increasing the range of that
context?  Acquisition is currently a rule-by-rule process, when it really ought
to be day-by-day or instructor-by-instructor.

	Experience with a number of experts may provide answers to these
questions and indicate where tools may be needed for the more mundane, but
more immediate, problems that arise in dealing with large amounts of
knowledge.

\index{production rules}The use of production rules
also poses certain problems, one of
which arises from the impact of rules on the organization of knowledge and
the style of program writing, especially as compared to a procedural view.
Rules are a reasonably natural and convenient form of knowledge encoding
for what may be termed ``single-level'' phenomena---it is easy to think of
single decisions or actions in terms of a rule.  Experience has
demonstrated, however, that even experts acquainted with the production
rule encoding tend to think of a sequence of operations in procedural
terms and find flowcharts a convenient medium of expression.  While
flowcharts can always be converted to an equivalent set of rules, the
conversion is nontrivial and sometimes requires reconsidering the
knowledge being expressed, since the two methodologies offer different
perspectives on knowledge organization and use.

	We may as well bow to the inevitable, then, and consider ways of
formalizing this translation process.  How might an expert describe a whole
{\sl sequence} of decisions, and how might the system then translate this
into an equivalent (unordered) {\sl set} of rules?\ffnote{Or a set of rules, plus a
meta-rule for ordering.  This approach assumes that, when feasible, it is
more profitable in the long run to use a single representation (in this
case, rules) and provide ways to translate all knowledge into this form,
than to introduce new representations, each tailored to a specific task.
Experience with {\TEIRESIAS} so far suggests that this is true, but the issue is still far
from settled.} This appears to be an interesting prospect for further work.

	We have also begun to encounter a standard technical problem, as
the combination of {\TEIRESIAS} and {\sc MYCIN} has grown beyond the resources
available in present PDP-10 based TENEX {\sc INTERLISP} systems.  The
combination is already large enough that many data structures have to be
retrieved from the disk during execution rather than kept in core.  While
this is accomplished with a very efficient hashing routine\fnote{The routine was
provided by Larry Masinter of Xerox PARC and Bill van Melle of the 
{\sc MYCIN} group.} the program is already too slow with normal machine loading---it
is caught on both ends of the speed-space trade-off.  Possible
solutions lie both in increases in hardware resources and  the use of faster
programming languages. 

	Finally, this report started with the suggestion that it would be
advantageous to replace the assistant indicated in Figure 1--1 by a
program that allows the expert to educate the performance program directly.
This offers many advantages from the point of view of speed, manpower, and
ease of knowledge base construction.  But what kinds of problems might it
produce?  Experience has demonstrated that a certain period of
acclimatization is necessary before new experts are familiar enough with
the {\sl weltanschauung} implicit in the performance program to be able to
view the domain and its challenges in those terms.  During that period, the
task of the assistant becomes more than simply translating the clinician's
statements into {\sc LISP} rules; it  includes re-interpreting them as
well.  If the expert is put in direct contact with the program, what
problems might arise from (unrecognized) differences in their models of the
world?  How much of what the assistant typically does in mediating between
the expert and system involves a shift in representations and a resolution
of such conflicts?  Can this ability be formalized and made a part of the
system itself?  It appears to be difficult, requiring, as it does, solutions to
problems known to be difficult, and will require a good deal of further
work.

\sectionskip
\sectionbegin[4]{The other themes; some speculations}
	Here we return at last to the  list of themes given
in chapter 1.  Each is described in terms of three topics: 
(a) meaning---the
substantive concept behind the catch\-phrase is explained and its utility made
clear; (b) examples---several examples are given of how the idea has been
employed in {\TEIRESIAS}; and (c) evaluation and prognosis---the examples are
evaluated in terms of the goals suggested by the phrases,
and an indication is given of what difficult problems remain.

	First, a word of warning.  Despite attempts to linearize this discussion, it remains a
network of interrelated ideas.  The network is densely interconnected, and most
of the ideas will lead to most of the others if followed far enough.  To make
the discussion comprehensible, many of those connections have been ignored, except where
truly necessary.  Even so, the discussion occasionally loops back on itself.

	It also tends to raise a good many more questions than it answers.  After
pointing out how these ideas have been used throughout the program, it looks
forward and speculates about where some of this might eventually lead.

\subsectionbegin[4.1]{Why Write a Program:  Two Views}
	The first five themes are grouped together because they represent
different techniques involved in a certain style of programming.  To make
that style clear, consider two different, very broad characterizations of
situations in which computer programs might be written.

	Traditionally, programs have been written to solve specific
problems and obtain ``an answer.''  The answer may be of many types,
including numeric (as in business and scientific computation), symbolic
({\eg}, information retrieval), or others of a more abstract form. The main
issue here is that these programs are applied to tasks with two important
characteristics:  (1)  There is ``an answer'' to the problem, and (2)  it is
possible to speak of a ``final,'' debugged version of a program that is to be
used extensively as it is, with relatively long periods between changes to
it.  This view encourages a style of programming in which the programmer
spends a long time thinking about the problem first, tries  to solve as
much of it as possible by hand, and then abstracts out only the very
end-product of all that thought to be embodied in the program.  That is, the
program can become simply a way of manipulating symbols to provide ``the
answer,'' with little indication  of what the original problem was or,
more important, what knowledge was required to solve it.

	Large, knowledge-based programs appear to have a fundamentally
different character.  As noted earlier, the construction of large knowledge
bases is a long-term operation that is never really ``finished.''  Not only
is the approach to competence incremental (and occasionally asymptotic),
but the fields to which such programs are applied are typically those which
are still under active development.  The knowledge base is thus inherently
a dynamic structure.  The aim here is thus not simply to build a program
that exhibits a certain specified beha\-vior,\penalty2000\ffnote{Indeed, it is not even clear
whether a behavioral definition of such programs makes sense.  In other
cases behavioral definitions have been proposed and used ({\eg}, for
automatic programming as in [Shaw75]).  They make sense in that case
because they are a more compact expression than the program that produces
the behavior.  It is not clear that this would be true of large knowledge
based programs---it appears difficult to give a complete specification for
their behavior in any terms that produce a definition smaller than the
actual knowledge base.} but  {\sl to use the program construction process itself
as a way of explicating knowledge in the field, and to use the program text as a
medium of expression of many forms of knowledge about the task and its
solution}.  That is, the program becomes more than a collection of symbol
manipulation instructions.  It is used as an environment in which to
elicit, collect, and store large amounts of diverse forms of knowledge.

	To sum up these two long statements in a single phrase:  In one case
it is reasonable to think in terms of a program that represents a complete
solution to a problem, and hence encode only the minimally necessary symbol
manipulation steps in that solution; in the other, the process is a
continuing one of codifying and accumulating information.

	This second view has been adopted in building the knowledge bases needed
for several performance programs over the years and has made it possible to organize
and represent the large amounts of knowledge required.  Question:  Is it
possible to turn the idea in on itself and apply it to the internal world,
the world of representations and data structures inside the program?  Will
it be as useful at the meta-level as it was at the object level?  Can it
help to assemble, organize, and maintain the large amount of knowledge about
representations and data structures that is necessary for knowledge base
construction and maintenance?  Other efforts in this vein have revolved
around the use of more formal methods ({\eg}, [Suzuki76], [Spitzen75]).
Will the essentially informal, heuristic techniques we want to employ offer
sufficient power?

	The attempt to answer these questions affirmatively has been one of
the central themes of this work.

	The immediate application of these ideas has been to provide a
basis for the tools described in earlier chapters.  More fundamentally,
however, they offer the possibility of getting a step closer to that
seductive promise of the stored program computer: introspection.  As we
have seen, the most useful forms of meta-level knowledge are those that
make it possible for the system to examine its own knowledge directly.
Every step removed from direct examination represents a level of defeat
and loses some of the advantages.  Thus the rule models are derived
from
rules directly and adjust to changes in the knowledge base, while the
explanation routines require rewriting whenever the control structure
changes, because they do not directly reference the code of the system they
are trying to explain.

	With this overview, let's consider some of the specifics and see
how the first five themes suggest steps in the direction of these goals.

\dimsectionbegin{Theme 1: Task-specific high-level languages make
code easier to read.}
``Task-specific high-level language'' refers, simply enough, to a
language in which the conceptual primitives are task specific.  The primary
example of this that we have seen is the set of rules, composed of primitives
like {\sl attribute} and {\sl value}, whose instances in {\sc MYCIN} include
culture {\tt SITE} and {\tt BLOOD}.  The rules were discussed in these terms
in Section 2--4.4, where it was noted that they are in effect a 
domain-specific high-level language with a rule as the sole statement type.

	There are several other examples, many of them found in the
schemata described in chapter 6.  Included are the ``language'' of
data structure {\tt RELATIONS}, the set of {\sl slotnames} of which the
schemata are composed, and the several types of {\sl advice} found in slots.

	The foremost reason for designing and using ``languages'' like these
is to make possible what we might label ``top-down code understanding.'' 
Previous efforts at building program understanding systems have been aimed
at several goals, including proving programs correct (as in [Waldinger74]
and [Manna69]) and for use in automatic programming (as in [Green74]). Most of
these systems attempt to assign meaning to the code of some standard
(domain independent) programming language like {\sc LISP} or
{\sc ALGOL}.

	Since the problems encountered in doing this are known to be
difficult, we have used the high-level ``languages'' as a convenient
shortcut.  Rather than attempting to assign formal semantics to ordinary
code, a ``meaning'' is assigned to each of the primitives in the language
and represented in one or more informal ways.  For example, part of the
meaning of the concept {\sl attribute} is represented by the routine
associated with it that is invoked during new rule acquisition (see 
Section 4--4.2).  Another part is represented by the code in the
explanation routines that checks the attribute in a premise clause when
dividing a rule premise into known and unknown clauses.  The inequality
described in Section 3--7 is another example; it is part
of the ``meaning'' of a predicate function.

	Since this technique has been employed extensively here, it is
important to consider its limitations.  While it does make
program understanding easier, it approaches the task at a higher conceptual
level, which makes the result correspondingly less powerful.  We cannot, for
instance, prove that the implementation of {\tt SAME} is correct, but we can use
its indicated ``meaning'' in the ways illustrated.  More fundamentally, the
entire approach depends on the existence of a finite number of ``mostly
independent'' primitives.  This means a set of primitives with only a few,
well-specified interactions between them.  The number of interactions
should be far less than the total that is possible and interactions that
do occur should be uncomplicated.  In the case of the rule language, for
instance, the concepts of certainty factor and attribute are independent,
while attributes and values have a well-specified relation that is
reasonably simple to represent.

	This is an important limitation, one that we have encountered
earlier.  Recall the discussion in Section 3--9 on the
deficiencies of the explanation routines that arise from the inability to
represent control structures.  We pointed out there the utility of a
language of ``intentions'' in making it possible for the system to explain
its actions, yet were unable to propose one.

	When it is possible to find such a language, however, it can be
applied in a variety of ways.  In the simplest case, the ``meaning'' of each
of the primitives is considered and used individually.  A slightly more
sophisticated use allows combinations of primitives to describe other
structures.  Consider, for example, the templates associated with each
predicate function.  The ability to dissect a given function call could
have been based on individual, hand-tailored routines associated with each
function.  Instead, a lower level of detail was used.  This makes it
possible for each function to carry information describing its own calls
and makes the system extensible:  As long as calls to a new function can be
described in terms of the available primitives, the function can be added
to the system without any change to existing facilities.

	This idea of ``top-down code understanding'' is the primary technique
we have used to provide a degree of introspection.  The system can examine
parts of its own structure and can ``understand'' them in terms of the
primitives of the high-level languages used.

\dimsectionbegin{Theme 2:  Knowledge in programs should be explicit and accessible.}
Con\-si\-der the following two lists, taken from the current
{\sc MYCIN} system:
\centerit{(BLOOD BONE BRAIN CSF JOINT LIVER LUNG MUSCLE TENDON-SHEATH)}

\vskip-24pt

\centerit{(RULE012 RULE146 RULE087 RULE234 RULE043 RULE101)}

	The first appears to be ordered alphabetically; the ordering, if
any, of the second is unclear.  To avoid introducing bugs when adding a new
element to either of them, two things are important: (a)  If the list is
ordered, what is the ordering criterion? and (b)  What is the significance
of the ordering ({\ie}, who depends on it)?

	The answer to the first question is required in order to determine
where in the list the new element is to be placed. The second is important
if we wish to consider violating the established order:  It gives a clear
picture of the consequences and indicates where changes may have to be
made if serious problems  result.

	In the case of the first list, the ordering is in fact alphabetic,
but this has no significance whatever to the rest of the system.  The
second list has a partial ordering of its elements that is crucial to
system performance.  Until recently, neither piece of information was
represented anywhere in the system (nor in fact in any documentation). The
same phenomenon manifests itself in many forms in almost all large systems.
There is typically a large share of folklore about all kinds of structures
that resides only in the minds of the system designers, yet such
information is often crucial to the process of changing anything in the
system.

	This is the kind of information we have tried to capture with a
variety of mechanisms. The first list above, for example, is classified
internally as an alphabetically ordered linear list, and its ordering
information is embodied in the updating function associated with the schema
for that data type.  The ordering of the second list can be expressed with
meta-rules and, as noted in chapter 5, this makes the problem of updating
disappear.  The list can be stored unordered, new rules can simply be added
to the front, and the meta-rules will take care of reordering the list.

	There are other examples as well.  The schema network, for
instance, makes explicit the interrelationships of the data types in the
system.  Note that the point is not that this particular approach with its
generalization hierarchy is necessarily the way the information should be
represented.  We claim only that information about the internal
organization of data types should in some fashion be organized, formalized,
and represented in program-accessible terms.

	The meta-rules offer a more general example of the theme.  One of
their primary contributions was to provide a way of making explicit what
most programming languages leave implicit---the criteria for selecting the
next know\-ledge source to be invoked.

	A final example worth recalling is the restriction on rule ordering
required for the ``self-referencing'' rules described in Section 7--4.4.
To maintain the commutativity of certainty factors, all
non-self-referencing rules must be invoked before any that are
self-referencing.  This important piece of information was represented
first in an obscure piece of recursive code buried in the control
structure, later in a separate function, and finally was captured by a
meta-rule which expresses the necessary ordering in a simple and
straightforward form.  It can be explained using the explanation
facilities and changed, if necessary, using the existing rule editor.  It
has thus progressed from being implicit and inaccessible to being more explicit and
more easily retrieved.

	Many problems remain, however.  The basic issue here is the
standard difficult problem of representation of knowledge.  One indication
of a shortcoming in our solution is hinted at by the comment above that we
have used a ``variety of mechanisms.''  Our informal approach is too
ad hoc and  needs to be more rigorous before it can claim to
be a substantive solution.  A far better solution would provide a single
representation formalism that could express all the different forms of
information required.  Some work ({\eg}, [Suzuki76]) has examined the
applicability of predicate calculus to similar problems, but that approach
has not yet developed techniques capable of dealing with the size and
complexity of the axiomatization and proof procedures required.

\dimsectionbegin{Theme 3:  Programs can be given access to and an
understanding of their own representations.}
One of the oldest and most fundamental issues in AI is that of
representation.  Different approaches have come and gone and generated
innumerable discussions of respective power and virtue.  But in all these
arguments, one entity intimately concerned with the outcome has been left
uninformed:  the program itself.

	Several capabilities follow from giving a program the ability to
examine its own representations.  The one we have employed most heavily is
the ability to make multiple uses of the knowledge in a single
representation.  Rules, for example, are:

\listbegin
\numlistentry[(a)]{viewed as code and executed to drive the consultation,}
\numlistentry[(b)]{viewed as data structures, and dissected and abstracted to form
the rule models,}
\numlistentry[(c)]{dissected and examined to produce explanations,}
\numlistentry[(d)]{constructed during rule acquisition, and}
\numlistentry[(e)]{examined and reasoned about by the meta-rules.}
\listend

As another example, the schemata
provide a basis for accessing and ``under\-stan\-ding'' many of the other
representations used in the system and support acquisition, as well as
storage and retrieval.  Each of these alternate uses requires knowledge in
addition to that found in the representation, but if the knowledge stored
there can be decoded, it need not be represented in more than one form.

	It is important to note here that the feasibility of such multiplicity
of uses is based less on the notion of production rules per se, than on the
availability of a representation with a {\sl small grain size} and a {\sl simple
syntax and semantics}.  ``Small,'' modular chunks of code written in a simple,
heavily stylized form (though not necessarily a situation-action form) would
have done as well, as would any representation with simple enough internal
structure and of manageable size.  The introduction of greater complexity
in the representation, or the use of a representation that encoded significantly
larger ``chunks'' of knowledge, would require more sophisticated techniques for
dissecting and manipulating representations than we have developed thus far.
But the key limitations are size and complexity of structure, rather than a
specific style of knowledge encoding.

	It is interesting to consider this view in the light of historical
developments.  When computers were programmed with plugboards, program and
data were clearly distinct.  With the advent of the stored program computer
to make it possible and {\sc LISP} to make it easy, the idea arose that
{\sl programs could be considered data}.  In these terms, the view proposed
here adds to this the ideas that

\listbegin
\numlistentry[(a)]{data structure architecture ({\ie}, the schemata) can
also be data, and }
\numlistentry[(b)]{control architecture (invocation criteria, the meta-rules)
can also be data. }
\listend
\noindent
As a result, the system gains access to two more of its fundamental components.

	This perspective underlies much of our work in knowledge
acquisition and knowledge base management and seems to have potential for
future application.  For example, the schemata can be viewed as definitions
of data structures, and that set of definitions might be examined for
inconsistencies.  Do two different structures make conflicting assumptions
about the organization of a third structure?  Are there loops, or
other illegal topological forms in the schema hierarchy?  This problem is
difficult because it means, first, defining what ``inconsistent'' means for
our informal knowledge representation and then devising effective
procedures for discovering it.

	More speculatively, might the system be able to discover or
conclude any ``interesting things'' in examining its own representations?
Two primitive forms of this exist currently in {\TEIRESIAS}:  The rule models indicate
statistical regu\-la\-ri\-ties found in the knowledge base, and the schema
network editor is able to suggest possible interconnections for a new
schema, based on its examination of the existing network.  Can more
sophisticated forms be developed?  Can {\TEIRESIAS} conceivably make useful
suggestions about representation design, by noticing, perhaps, that (a)  two
representations are very ``similar,'' (b)  there are relatively ``few''
instances of either, and (c)  combining the two might make
the schema network ``simpler.''  While
this would have great utility in dealing with the
organization and structure of large programs, the difficulty starts with
attempts to give precise definitions to the words in quotes, and multiplies
quickly.  But let us speculate just a little further.  Is it possible that
some of those suggestions from {\TEIRESIAS} about data structures might in fact have
implications for understanding the domain itself?  Can the regularities
discovered by the rule models, for instance, ever suggest new and
interesting things about the structure of reasoning in the domain?  Or can
the system's ability to discover useful things about data structures be
made sophisticated enough that its suggestions offer some help in forming
theories about the domain?

	These are difficult issues.  They address the problem of pattern
detection and theory formation applied to a slightly different domain---the
domain of representations.  Chapter 5 considered this issue briefly and
examined the rule models as an example of concept formation.  All of the
hard problems noted there are yet to be solved, as are the more complex
issues involved in theory formation.  The examples found in the present {\TEIRESIAS}
system serve basically as demonstrations of feasibility.

\dimsectionbegin{Theme 4:  Programs can be self-understanding.}
One aspect of the idea of ``self-understanding'' has been illustrated
in the previous section and concerns a program's ability to understand
itself in terms of representations and data structures.  Here we consider
how it might understand its own behavior.

	To make the discussion more concrete, consider the following
criteria for ``understanding'':

\listbegin
\unnlistentry{Can the program}
\nestlistbegin
\bullistentry{explain its behavior?}
\bullistentry{explain what it knows?}
\bullistentry{change its behavior?}
\bullistentry{change the content of its knowledge?}
\nestlistend\listend

	These criteria can be met with varying levels of success with respect
to completeness ({\ie}, how much of the system can be explained or changed)
and generality ({\ie}, are the techniques specific to the current
implementation or are they more broadly applicable).

	Consider the issue of generality first, and examine the combined
{\TEIRESIAS}-{\sc MYCIN} system as one example.  Chapter 3 noted the variety of
techniques that has been used to generate explanations.  These ranged from
one program written specifically to explain part of another, to the
system's ability to use the templates to examine the same rules it was
executing.  This latter form of introspection is the most powerful of the
solutions we have explored, since it adjusts automatically to changes in
the program's knowledge base and control structure (at least the part of
the control structure that can be captured in meta-rules).

	The question of completeness for the first three of the criteria
above is answered currently by considering how much knowledge is in rules.
The system can explain behavior that results from rule invocation, can
refer to knowledge embedded in rules, and can change its behavior by the
acquisition of new rules.  Where knowledge or behavior is not rule based,
it cannot currently be explained.  (The ability to acquire new conceptual
primitives via the schemata offers another foundation for acquisition of
new knowledge, so the system is currently more complete with regard to
changing the content of its knowledge.)

	There appears to be a fundamental limit on the completeness with
which program behavior and knowledge might plausibly be explained or
modified.  It is characterized in the introduction to [Minksy68] as the
distinction between ``compiled'' and ``interpreted'' behavior, where it is
suggested that some behavior, some knowledge, may reasonably be considered
``compiled'' and, hence, inaccessible to further decomposition.  The issue is
more than just choosing a convenient level at which to view system
behavior, as we did in chapter 3.  Some behavior and knowledge is simply
going to be inexplicable.  For {\TEIRESIAS}-{\sc MYCIN}, for example, this might
mean that the system would not and, indeed, need not ever be able to
explain things like the mechanism of rule retrieval.

	In line with our emphasis on making knowledge explicit, we might
add another possibility:  In addition to being compiled or interpreted, knowledge
might also simply be absent.  A program clearly cannot explain what it
doesn't know, and before the use of meta-rules, {\sc MYCIN} had no way of
explaining why certain rules were executed in a particular order.  It was
in a position roughly analogous to someone who can recall a list from rote
memorization, but never learned the rationale behind its order.\ffnote{See
[Parnas75] for some further thoughts on ``compiled'' and ``interpreted''
knowledge and its use in system design.}

	As noted earlier, our efforts toward explanation, and this larger
goal of self-understanding, have really only begun.  The issue is again one
of representation: How can knowledge be represented in forms that allow it
to be both used and dissected?  Our use of rules as a high-level language
has provided a certain level of performance, but as previous sections have
noted, there are many problems yet to be solved.

\dimsectionbegin{Theme 5:  A representation can usefully be more than a
densely encoded string of bits.}
\index{knowledge base management}There is an obvious
corollary to the belief that a program can be
more than the minimum set of symbol manipulation operations necessary for
solving a problem.  Analogously, a representation can be viewed in terms
other than the minimum number of bits that will carry the information
required.  This becomes necessary if we want a program to modify its
knowledge base without introducing errors:  If the program is to manipulate
its representations in intelligent ways, then it needs to know a good deal
more about each than standard declarations typically supply.

	Some of the additional information about representations we used
includes:  its structure, the set of legal values for each component of the
structure, its interrelations with other representations (as indicated by
the {\tt RELATIONS}), and its place in the whole collection of data
structures (as indicated by the schema hierarchy).  All of this information
is necessary to perform even the simple and straightforward ``clerical''
kinds of tasks involved in knowledge base maintenance.

	In addition, it makes possible much of the ``intelligence'' displayed
by the acquisition system.  As noted above, the representations used in the
knowledge base are both data structures and a model of the domain of
application.  Part of an ``intelligent'' process of acquisition involves
being able both to use this information and to insure that it is properly
maintained.  Chapter 5, for example, noted that one of the consistency
constraints used in generating interpretations of a new rule relied on this
world model to insure that an attribute in a clause was paired with an
appropriate value.  The integrity of the model is maintained by devices
like the {\tt RELATIONS} list, which insures that new structures are properly
integrated and  that existing dependencies are maintained.

	All of this means an emphasis on the careful representation and
organization of this information.  The schema language is one
attempt to provide a framework for organizing and accessing this
information and  may help to make its expression easier.

\null
\index{meta-level knowledge}These first five themes
have explored how {\TEIRESIAS} supplies a base of
meta-level knowledge---information about the organization, structure, and
use of knowledge in the performance program.  The ultimate goal is to make
possible a useful level of introspection, one that will support
``intelligent'' behavior for the acquisition, explanation, and effective use
of object-level knowledge.  {\sl Task-specific high-level languages }are the
primary mechanism used to make this feasible, since they avoid the
difficult problems involved in having a program read ordinary code or
examine standard data structures.  The emphasis on {\sl making knowledge in
programs explicit and accessible} follows from the desire to have the
program modify its own knowledge base and behavior.  To avoid introducing
errors in doing this, it needs a large store of {\sl information about its own
representations}, information of the sort that is typically left either
informally specified or omitted entirely.  The concept of a {\sl program that
``understands'' itself} concerns its ability to account for, and modify, its
behavior and knowledge; this is introspection at the behavioral level.
Finally, the suggestion was made that a {\sl representation can be viewed in a
broad context}, both as a data structure in the program and as part of
the program's model of the world.  This context is used as the basis for a
language and framework in which to organize and express much of the
required knowledge.

\dimsectionbegin{Theme 6:  A program can have some grasp on its own complexity.}
\index{complexity}The thought has been around for some time that a
program ought to
be able to help out in dealing with its own complexities (a recent
collection of suggestions along this line is in [Winograd74]).  There are
at least two ways in which it should be possible to enlist a program's
help.

	The first involves simply coping with the complexity that
inevitably arises.  It is, as we have seen, difficult to make a single
small change to any large system. There are typically a number of related
changes that have to be made and constraints that must be met.  Rather than
relying on memory (or documentation) to keep track of all the details, why
not have the system do it? Whatever knowledge there is concerning those
details ought to be made explicit and then made accessible to and
comprehensible by the system.  Several examples of this have been explored
in earlier chapters.  The use of the schemata for acquisition of new
conceptual primitives is a primary example.  The schemata were designed to
confront exactly this problem of the complexity and detail of structure and
interrelationships typically encountered in any large set of data
structures.  Another example is found in the suggestion that it ought to be
possible to edit the schemata and have {\TEIRESIAS} edit all the instances similarly.
The use of a ``totally typed'' language during acquisition is still another
example.  As illustrated, this makes it possible for the system to specify
to the expert the legal form of the answer to any of its questions and to
check the responses for syntactic validity.  Finally, even the simple recap
provided by the explanation routines can help the user cope with the wealth
of detail that arises in a long consultation.

	The second form of aid might arise from the capability of the
system to simplify itself for the benefit of the viewer.  Introspection may
mean little if a program is very large and its behavior complex.  If the
system ``explains'' itself at a level that leaves the viewer drowning in a
wealth of detail, little has been accomplished.  Is it possible, then, for
a program to abstract that detail and present a comprehensible picture? 
Can the ``magnification'' be varied to provide accounts which offer different
levels of details?

	Some very basic steps in this direction were described in chapters
3 and 5.  The ``higher level explanations'' available with the {\tt WHY \<n>}
command, for instance, offer a very simple form of simplification and
variable magnification.  The rule models are another example:  As abstract
descriptions of the content of the knowledge base they help to make clear
the general trends in the reasoning while suppressing less important
detail.

	But these are only first steps.  A more sophisticated solution
would make it possible to base different explanations on different models
of the underlying processes.  It might even be able to choose a model
tailored to both the current sequence of events and the current viewer.
There is clearly much work that remains to be done.

\dimsectionbegin{Theme 7: Programs can be self-adjusting.}
\index{complexity}As the previous discussion suggests,
dealing with complexity is
difficult.  One approach to the problem views it as an issue of design and
asks:  Rather than creating tools to recapture programs whose complexity
has gotten out of hand, is it instead possible to design programs in ways
that avoid complexity from the outset?  That is, can we try to partition
problems and design representations in ways that make them simple and
self-contained.  A few relevant ideas have been in the air for some time.
Simon's {\sl Sciences of the Artificial} contains some very general
observations on system decomposition, while {\sl Structured Programming}
[Dijkstra72] offers some thoughts on control structure design.

\index{insulation}There are also some
well-established techniques of programming
which confront this problem.  One approach emphasizes dealing with change
by {\sl insulating} one part of the program from another.  A number of
techniques have been used, and a somewhat larger number of names have been
invented for them.

\listbegin
\numlistentry[1]{{\sl Indirection}.  Indirection insulates the program from referential
changes.  Thus,  indirection in code (indirect addressing) makes possible
compile time (or even execution time) assignment of addresses.  Indirection
in data ({\eg}, use of a pointer to a set rather than an explicit list of
its members) offers a similar flexibility.}

\numlistentry[2]{{\sl Records}.  One level of record structures will insulate a program from
changes in data structure implementation (``dataless [representation-free]
pro\-gram\-ming'' [Balzer67]).  Two levels of record structures can be used to
insulate a program from certain kinds of changes in data structure design
(``data independence,'' [Bachman75]).}

\numlistentry[3]{{\sl Data abstraction}.  Where records define a representation by its
structure, data abstraction defines it by its behavior.  It offers a
similar form of insulation, allowing code to refer to data structure
behavior without regard to details of the implementation of that behavior
(``abstract data types'' [Liskov74]).}

\numlistentry[4]{{\sl Decomposition}.  By choosing the proper boundaries, systems can be
decomposed into modules that are insulated from changes in one another
(``infor\-ma\-tion hiding'' [Parnas72]).  (Consider the discussion about
``traffic directors'' in Section 6--9.1 in these terms.)}

\numlistentry[5]{{\sl Content-directed invocation}.  By describing (rather than naming)
know\-ledge sources (KSs) and by effecting that reference via direct
examination of KS content, system behavior is insulated from changes to the
code of any of its KSs.  Thus a KS can be edited and its pattern of
invocation will change accordingly, without the necessity of making changes
elsewhere in the system (see the discussion in Section 7--5.3.3).}
\listend

\index{self-adjusting systems}There may quite plausibly,
however, be problems with elements that
are inherently tightly connected and complex.  It may be impossible to deal
with these without a corresponding degree of complexity in the program that
solves them.  Rather than attempting to avoid complexity, therefore, can we,
in this case, design systems which are {\sl self-adjusting}?  That is, is it
possible to design a system to be inherently flexible, rather than as
drastically fragile as  most programs typically are?

	Achieving such flexibility would require  systems that are
inherently stable, in which the effects
of changes are easily accommodated.  Many large programs
are currently similar to a house of cards, in that changes or additions
must be made with utmost care, lest the entire structure collapse.  There
are simply too many unknown, unstable, and critical interdependencies.
What we would like is something more akin to a pond---throw in a rock or
add a lot of water and there is a commotion initially, but eventually,
the ripples subside, the ``system'' adjusts to the new conditions by
expanding or reconfiguring itself and remains as ``competent'' a pond as
before.

	One established programming technique that confronts the problem of\index{transform}
adjusting to changes is the idea of a {\sl transform}.  It expresses and
automatically takes care of necessary interrelationships by propagating the
effects of changes.  Consider a compiler in this light.  To see the point
most clearly, imagine in its absence the difficulty of maintaining both
source code and machine code for a program.  After any editing in the
source, it would still be necessary to edit the machine code analogously.
But given the subtlety, complexity, and amount of work involved in just
understanding the mapping from source to machine language, the transform
has been specified precisely and turned over to the system.

	An  analogous situation exists with respect to data structures.
There are programs with established, complex interrelationships of data
structures, and, at very best, the relationships are only documented
somewhere.  The programmer still does multiple edits of data structures to
maintain those interrelationships, when he should be able to edit just one
structure and ``recompile the data structures'' to have the system take care
of the ``ripples.''  It is true that compilers are both better understood and
far more stable as transforms.  Data structure interrelationships are both
invented anew for each program and  subject to constant change as
programs evolve.  Even so, we claim that there are interesting and
important results that may follow from specifying the interrelationships.
First, it can conceivably make possible larger systems than might otherwise
be maintainable, and second, the task of being precise about those
relationships can help insure that the design is free from obvious
inconsistencies.

	Our system has something of this sort in the {\tt RELATIONS}
associated with each schema.  They specify data structure
interrelationships and offer one level of self-adjusting character:  The
effects of adding a new instance of a schema are propagated
through the rest of the system.  Recall, for example, the extensive
interaction triggered by adding a new culture site to the knowledge base of
the performance program.  To push this one level further, however, consider
a more fundamental change.  What if it were necessary to change the
definition of a data structure?  Part of this might include writing a whole
new set of {\tt RELATIONS} specifications for it and making modifications to
the {\tt RELATIONS} for other structures.

	But is there an easier way to do this?  Would it be possible to
write a definition of the data structure and ``compile'' it into the
appropriate set of {\tt RELATIONS}.  Recall, for example, the three
categories of culture sites mentioned in chapter 6.  It should be possible
to say that every culture site is either {\sl sterile, non-sterile}, or
{\sl indeterminant}, that is, that there are three mutually exclusive and
exhaustive classifications for it.  If, later, it were discovered that yet
another classification were necessary, rather than rewriting all the
{\tt RELATIONS} by hand, it should be possible to edit just the definition of
a {\tt SITE}, and have the ``data structure compiler'' recompute all the new
{\tt RELATIONS}.

	This would be a useful capability, since representation redesign is
an occupational hazard of programming and one that typically has
far-reaching effects for even small modifications.  Logical errors in
design also occur, and with more formal definitions it might be possible to
check for inconsistencies during the ``compilation.''

	Note that it also extends the concept of a self-adjusting system.
One level of change propagation was accomplished by the precise specification
of the data structure interrelationships.  Another level would be made possible
by deriving the interrelationships themselves from more basic elements,
making it possible to propagate the effects of changes in definitions.

	But all this is only speculation that raises more questions than it
answers.  What constitutes a ``definition'' of a data structure?  What
information should it contain?  The discussion above indicates that both
structural and behavioral definitions have been proposed; there may be yet
other possibilities.  How is the information to be represented?  Much work
has been directed toward the use of formal languages, and has demonstrated
the complexity that lies beneath apparently simple structures when their
properties are described precisely (see, for instance, [Spitzen75]).

	One further speculation to set all this in the broadest terms: 
Some elements of the techniques that have been employed in dealing with
change seem general enough to be widely applicable.  Can this set of
principles be extended and made large enough to provide useful guidance?
Might there someday be a ``science'' of representation design?

\sectionskip
\sectionbegin[5]{Projections}
The past seven chapters have explored some preliminary solutions to
the myriad of problems encountered in the attempt to facilitate the
construction of expert systems.  This is only a beginning, for much remains
to be done.  But what might be the eventual impact of success?  What might
be gained by solving the problems?  Two simple but significant
possibilities come to mind.

	One result might be the realization of a different form of generality
in problem solving.  This report began with the observation that the search
for generality in AI programs has traditionally focused on broadly
applicable problem-solving methods and has so far proved unsuccessful.  An
alternative form of generality was suggested, one that took its breadth not
from general problem-solving methods, but from powerful knowledge
acquisition techniques.

\extractbegin
The appropriate place for an attack on the problem of generality may be at
the meta-levels of learning, knowledge transformation, and representation,
not at the level of performance programs.  Perhaps for the designer of
intelligent systems what is most significant about human problem solving
behavior is the ability to learn specialties as needed---to learn expertise
in problem areas by learning problem-specific heuristics, by acquiring
problem-specific information, and by transforming general knowledge and
general processes into specialized forms.\ffnote{From [Feigenbaum71].}
\extractend

\noindent
We have taken a  step in this direction by making possible a limited
form of communication between an expert in an application domain and a 
high-performance program.  The program is capable of explaining significant
portions of its behavior and is prepared to accept new knowledge and
integrate it into its knowledge base.

	Finally, if it were truly easy to construct large knowledge-based
systems, then might they eventually become an important medium of
expression?  Might they provide a medium for the formalization of knowledge
in domains where it is as yet highly informal, and could they provide a
useful framework for its expression and organization?  Such systems might help
to collect and organize knowledge on a scale large enough to become 
useful tools in the attempts to  understand and develop theories for domains
where no cohesive theories yet exist.

\vfill
\null
\epigraph{I will say nothing further.}\vskip -9pt
\epigraph{Against this answer let your temper rage as wildly as you will.}
\author{Sophocles}
\source{Oedipus the King, lines 343--344}

\worldend